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1.  Project  1:  Simultaneous  EEG  and  NIRS  Tomographic  Imaging  Based  on 
New  Electro-Optodes 


1.1  Authors 

The  authors  on  this  project  are  Tzyy-Ping  Jung,  Jeng-ren  Duann,  and  Jin-Chern  Chiou  from  the 
Institute  for  Neural  Computation,  University  of  California,  San  Diego,  La  Jolla,  CA. 

1.2  Introduction/Objective 

This  study  aims  to  design,  fabricate,  and  test  a  truly  portable,  lightweight,  noninvasive, 
neuroimaging  system  that  supports  simultaneous  electroencephalo graphic  (EEG)  and  functional 
near-infrared  spectroscopic  (fNIRS)  acquisition  to  provide  complementary  information  of  the 
brain.  The  acquired  neural  dynamics  could  be  comparable  to  those  obtained  from  simultaneous 
EEG  and  functional  magnetic  resonance  imaging  (fMRI)  recordings.  The  envisioned  dual¬ 
modality  neuroimaging  system  will  feature  our  newly  developed  EEG/NIRS  electrodes,  known 
as  electro-optodes,  based  on  micro-electromechanical  system  (MEMS)  technology  and 
miniaturized  supporting  hardware  and  software  for  biological  or  cognitive  monitoring  of 
participants  in  operational  environments.  Another  major  goal  of  this  project  is  to  develop 
advanced  signal-processing  methods  and  software  for  (1)  increasing  signal-to-noise  ratio  of  the 
acquired  EEG  and  optical  imaging;  and  (2)  two-dimensional  (2-D)  and  three-dimensional  (3-D) 
image  reconstruction  and  rendering  to  reconstruct  the  optical  images  for  investigating 
neurovascular  coupling  with  minimum  inter- modality  interferences  and  preparation  demands. 

The  advantages  of  simultaneous  EEG/fMRI  recordings  have  been  explored  by  many  studies  and 
received  increasing  attention  in  neuroscience  community  (Ives  et  al.,  1993;  Huang-Hellinger  et 
al.,  1995;  Lemieux  et  al.,  1997).  However,  simultaneous  EEG/fMRI  recordings  have  formidable 
technical  problems.  For  example,  in  EEG/fMRI  recording,  any  ferromagnetism  in  the  scanner 
can  be  dangerous  for  the  subject  and  will  certainly  cause  unacceptable  loss  in  the  blood-oxygen- 
level  dependence  (BOLD)  image.  Then,  small  movements  of  the  electrodes  or  cables  in  a  strong 
magnetic  field  will  introduce  artif actual  currents.  The  recorded  EEG  signals  will  be  completely 
useless  before  the  magnetic  resonance  (MR)  artifacts  can  be  removed.  To  remove  the  MR- 
induced  artifacts,  the  onset  timings  and  the  waveforms  of  the  MR  artifacts  need  to  be  precisely 
registered  during  the  acquisition  of  each  MR  image  slice.  This,  in  turn,  requires  high  sampling 
rate  for  EEG  recordings  (at  least  5  KHz  compared  with  250  Hz  for  regular  EEG  recordings).  On 
the  other  hand,  EEG  electrodes  may  also  introduce  deformation  of  the  magnetic  field  of  the  MR 
scanner  and,  in  turn,  cause  signal  void  in  (f)MR  images  around  the  scalp  electrodes.  All  these 
interferences  make  simultaneous  EEG/fMRI  and  post-processing  extremely  difficult.  Therefore, 
a  truly  small,  lightweight,  battery-powered  EEG/fMRI  system  for  concurrent  EEG/fMRI 
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recordings  of  the  freely  moving  subjects  in  real-world  environments  will  be  extremely  difficult, 
if  ever  possible,  in  the  near  future. 

This  study  is  integrating  a  novel  EEG/fNIRS  electro-optode  that  can  be  used  as  an  EEG 
electrode  and  a  near-infrared  (NIR)  light  emitter  and  photodetector,  supporting  wearable  data 
acquisition  (DAQ),  signal-processing,  and  image  rendering  software  into  a  wearable 
neuroimaging  system.  We  will  also  develop  advanced  2-D  and  3-D  image  reconstruction 
algorithms  to  reconstruct  the  optical  images  from  the  recorded  fNIRS  signals.  The  envisioned 
simultaneous  EEG/fNIRS  acquisition  system  is  completely  novel  and  different  from  any  other 
currently  available  systems.  First,  the  envisioned  system  features  brand-new  dry  EEG/NIRS 
electro-optodes,  which  can  function  as  an  NIR  emitter,  a  detector,  and  an  EEG  electrode  without 
requiring  any  skin  preparation  and  conductive  paste.  Most  importantly,  the  electro-optode  also 
comprises  built-in  NIR  light  sources  and  NIR  waveguide  to  bypass  the  high  reflectance  of  the 
human  dead-skin  layer  such  that  low-power  NIR  light  sources  will  be  sufficient  for  high-quality 
recording.  In  addition  to  the  innovative  hardware  design,  fabrication,  and  integration, 
Independent  Components  Analysis  (ICA)-based  signal  processing  can  separate  the  mixture 
signals  caused  by  multiple  light  sources  and  multiple  possible  pathways.  As  a  result,  we  are  able 
to  reconstruct  a  3-D  optical  tomography,  which  may  be  more  specific  to  the  underlying  BOLD 
effects.  In  turn,  we  should  be  able  to  better  assess  the  interrelationship  between  the  BOLD  effects 
and  the  neural  activities  measured  by  the  simultaneously  recorded  EEG  signals. 

1.3  Original  Approach 

This  research  proposed  to  design,  fabricate,  and  test  a  truly  portable,  lightweight,  noninvasive, 
neuroimaging  system  that  supports  simultaneous  EEG  and  fNIRS  acquisition  to  provide 
complementary  information  of  the  brain.  The  development  effort  was  proposed  according  to  the 
following  schedule: 

•  Year  1:  We  will  first  develop  EEG/fNIRS  electro-optodes  and  test  the  safety  of  the  newly 
developed  electro-optode.  The  specific  goals  are  to  (1)  observe  the  depth  of  penetration  of 
the  micro-spikes  to  see  if  they  are  staying  in  the  epidermis  layer  without  further  puncturing 
into  the  papillary  dermis  and  (2)  examine  the  tissue  damage  by  comparing  the  skin  slice  cut 
from  the  site  with  and  without  the  applications  of  electro-optodes. 

•  Year  2:  We  will  conduct  the  same  penetration  rate  study  listed  previously  on  human 
subject’s  fingers  but  use  the  proposed  electro-optodes  with  built-in  light-emitting  diode 
(LED)  light  source.  The  results  of  this  study  will  be  compared  to  those  obtained  from  the 
experimenting  using  external  NIR  light  sources.  This  comparison  will  help  us  to  determine 
the  minimum  power  level  needed  in  the  final  implementation  of  the  built-in  LED  light 
sources  of  the  electro-optodes. 
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•  Year  3:  We  propose  to  (1)  build  a  2-D  sensor  array  with  16  (4x4)  electro-optodes  by  first 
trying  to  arrange  the  16  electro-optodes  in  a  4  by  4  grid  with  emitters  and  detectors  placed 
in  an  interleaved  fashion;  and  (2)  establish  imaging  protocol(s).  Since  all  the  electro- 
optodes  can  be  used  as  an  emitter  and  photodetector,  the  montage  of  such  16-channel 
fNIRS  imaging  systems  can  be  extremely  flexible. 

•  Year  4:  (1)  We  will  apply  the  16-channel,  2-D  EEG/fNIRS  acquisition  patch  to  acquire 
both  electromyogram  (EMG)  signals  and  optical  images  from  the  human  forearms.  The 
4x4,  16-channel,  2-D  imaging  system  will  be  applied  to  the  inner  side  of  the  human 
subjects’  forearms  to  acquire  both  EMG  and  optical  images  when  the  subjects  are 
performing  finger  tapping  and  making-a-fist  tasks.  As  expected,  finger  tapping  or  making  a 
fist  will  involve  different  muscle  bundles  of  the  forearm.  The  muscle  activities  will  create 
electric  signals,  hence  EMG,  as  well  as  oxy-  and  deoxyhemoglobin  concentration  changes 
due  to  oxygenation  of  the  muscle  cells.  The  signals  accrued  by  the  envisioned  system  can 
be  used  to  investigate  the  relationship  between  the  underlying  muscle  oxygenations  and  the 
corresponding  electric  signals  from  the  combined  imaging  modalities.  (2)  We  will  apply 
the  16-channel,  2-D  simultaneous  EEG/fNIRS  imaging  system  to  the  human  scalp  with  a 
coverage  of  left/right  sensorimotor  cortex  (i.e.,  pre-  and  postcentral  gyri).  We  will 
simultaneously  acquire  EEG  and  fNIRS  images  on  the  scalp  from  the  human  subjects 
performing  the  same  finger  tapping  and  making-a-fist  tasks  as  in  Year  3.  In  the  recorded 
EEG  signals,  we  expect  to  find  the  motor-related  mu-rhythm  activities  as  can  be  found  by 
International  10-20  EEG  systems.  In  addition,  we  will  also  apply  the  2-D  image 
reconstruction  algorithm  to  the  recorded  fNIRS  signals  using  the  same  protocol  as 
described  previously  to  reconstruct  the  fNIRS  signals  acquired  from  the  human  scalp. 

•  Year  5:  We  plan  to  expand  the  coverage  of  the  simultaneous  EEG/fNIRS  image  acquisition 
to  the  whole  head  by  adopting  the  standard  10-20  EEG  system  montage.  This  work  allows 
us  to  (1)  expend  the  number  of  electro-optodes  from  16  to  32  channels,  (2)  co-register 
fNIRS  and  EEG  measurements  to  the  MR  structure  scans  of  the  individual  subjects  or  to 
the  standard  MNI  brain  template,  and  (3)  reconstruct  3-D  optical  images  for  the  whole 
brain  in  cognitive  studies. 

1.4  Results  to  Date 

1.4.1  Design  and  Development  of  Electro-Optodes  (EEG/fNIR  Sensors) 

Background.  Photoplethysmography  (PPG)  is  a  simple  and  low-cost  optical  technique  that  can 
be  used  to  detect  blood  volume  changes  in  a  micro-vascular  bed  of  tissue.  PPG  has  been  applied 
in  many  different  clinical  settings,  including  clinical  physiological  monitoring,  vascular 
assessment,  and  autonomic  function.  Since  PPG  is  a  non-invasive  method,  it  should  not  cause 
any  damage  to  subjects.  This  study  uses  red  and  NIR  light  to  assess  brain  dynamics  because  it  is 
very  cost  effective  and  easy  to  apply  and  use.  Further,  the  light  source  (LEDs)  and  detector 
(photodiodes  [PDs])  could  be  easily  embedded  on  a  portable  probe. 
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Major  Progress.  In  the  past  few  months,  we  have  explored  many  different  electronic  and  form 
factor  designs  for  the  electro-optodes,  which  can  simultaneously  measure  EEG  and  fNIR  signals 
After  numerous  simulations  and  tests,  we  have  finalized  the  form  factor  of  electro-optodes  and 
fabricated  some  samples  for  testing.  This  study  has  also  developed  an  innovative  mechanism  to 
place  the  sensor  to  the  scalp  to  minimize  the  interference  from  the  hair  to  the  recording. 


Design.  Figure  1  shows  a  schematic  diagram  of  the  O-shaped  PPG  sensor.  In  order  to  detect  PPG 
signal,  there  can  be  one  or  several  LEDs  and  photodiodes  on  this  O-type  probe. 


Figure  1.  An  O-shaped  NIR  sensor  integrating  LEDs  and  PDs. 

Figure  2  illustrates  the  application  of  the  PPG  sensor  to  the  scalp.  By  snipping  a  small  strand  of 
hair  and  putting  them  through  the  hole  of  the  O-shaped  sensor,  the  hair  will  be  fixed  and  a  “ring” 
of  the  skin  will  be  exposed.  The  light  source  then  can  directly  illuminate  and  reflect  from  the 
subject’s  scalp  (as  shown  in  figure  2). 


PPG  Probe  A  bunch  of 


Figure  2.  The  application  of  the  PPG  sensor  to  the  scalp. 

Development  and  Implementation.  Figure  3  shows  the  printed  circuit  board  (PCB)  and  the 
LED/PD  of  the  electro-optodes.  The  inner  and  outer  diameters  of  the  electro-optode  are  1  cm  and 
2  cm,  respectively.  The  EEG  sensor  is  soldered  to  the  small  rectangular  area  shown  in  figure  3. 
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Figure  3.  The  final  form  factor 
of  the  electro-optodes. 

1.4.2  A  16-channel  Analog  Front-End  (AFE)  Amplifier 

To  acquire  EEG  and  fNIR  data  measured  by  the  electro-optodes,  we  have  recently  developed  a 
16-channel  analog  front  end  (AFE)  amplifier  in  the  past  year.  The  amplifier  provides  signal 
conditioning  and  filtering  of  the  weak  neural  signals. 

Design.  Figure  4  shows  a  block  diagram  of  the  complete  16-channel  amplifier  chip.  The 
differential  difference  amplifiers  (DDA)  are  used  as  first  stage  of  the  AFE  amplifier  for  high- 
common-mode  rejection  ratio  (CMRR)  and  low-noise  requirements.  A  16:1  multiplexer  (MUX) 
controlled  by  the  clock  signal  allows  all  16  DDAs  share  the  same  second  and  third 
amplifier/filter  stages.  The  MUX  frequency  is  set  to  200  kHz  to  scan  the  16-channel  inputs.  The 
second  stage  and  third  stage  use  operational  transconductance  amplifier  (OTA)  to  implement  a 
low-pass  filter,  high-pass  filter,  and  gain  amplifier  with  selectable  bandwidth  and  gain.  The  gain 
and  low-pass  filter  frequencies  are  controlled  by  digital  signals:  LPF_Select  [0:2]  and 
Gain_Select  [0:2]. 


5 


Figure  4.  A  block  diagram  of  a  16-channel  amplifier  chip. 

The  DDA  in  this  amplifier  is  modified  from  Ng  and  Chen  (2005),  and  the  circuit  is  depicted  in 
figure  5.  To  reduce  the  original  flicker  noise,  the  input  stage  (Ml -M2  and  M3-M4)  are  positive 
metal-oxide  semiconductor  (PMOS)  with  wide  width  and  operated  in  a  weak  inversion  region. 
The  input  stage  transfers  the  input  voltage  into  current.  M9  and  M10  construct  a  common  source 
amplifier.  The  differential  currents  flowing  through  M7  and  M8  are  transferred  back  to  voltage 
signal  by  the  load  device  Mil  and  M12.  The  output  gain  stage  of  DDA  is  constructed  by  the 
two-stage  amplifier  (M13-M19),  which  acts  as  a  differential-to-single-ended  converter  and 
buffer.  Cl  and  C2  are  Miller  capacitances  used  to  increase  the  phase  margin.  Finally,  the  DDA- 
based  noninverting  amplifier  is  implemented  from  the  topological  placement  of  R1  and  R2.  The 
input/output  relation  is  defined  as 

v„„, =(*,/*,+  ifc  (1) 
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Figure  5.  Circuit  schematic  of  the  DDA  used  in  present  neural  amplifier. 

Figure  6  shows  a  schematic  of  a  fully  differential  OTA  used  in  the  second  stage  and  third  stage  of 
our  16-channel  neural  amplifier.  The  transconductance  Gm  of  the  amplifier  is  controlled  by  the 
bias  current  from  M9.  The  input  transistors  (Ml -M2)  are  operated  in  the  weak  inversion,  and  the 
current  mirror  transistors  (M5-M8)  are  operated  in  the  strong  inversion.  This  design  minimizes 
input-referred  noise  for  a  given  bias  current  (Harrison  and  Charles,  2003).  Figure  7  shows  the 
schematic  of  the  second  stage,  which  serves  as  a  programmable  low-pass  filter.  The  filter  is 
implemented  by  two  series  PMOS  transistors  across  C2  operated  in  subthreshold  region.  The 
mid-band  gain  AM  is  set  by  Cl/C2=40.  The  cutoff  frequency  is  approximately  defined  as 
Gm/(AMCL),  where  Gm  is  the  transconductance  of  OTA.  The  selectable  on-chip  capacitive  loads 
provide  50  Hz  to  10  kHz  lowpass  cut-off  frequency. 
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Figure  6.  Schematic  plot  of  OTA  used  in  the  second 
and  the  third  stages. 


Figure  7.  The  schematic  of  the  second  stage  with  selectable  cutoff  frequency. 

Figure  8  shows  the  third  amplifier  stage  with  tunable  gain  and  fixed  high  cutoff  frequency.  The 
ratio  of  capacitors  determines  the  gain  factor.  The  two  series  PMOS  resister  across  a  capacitance 
is  employed  to  provide  the  highpass  cut-off  frequency  of  0.1  Hz.  The  final  output  of  the  AFE 
chip  is  serial  data,  of  which  the  packaging  volume  and  wire  bonding  pads  can  be  reduced  due  to 
fewer  input/output  (I/O)  numbers. 
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Implementation  and  fabrication.  The  16-channel  neural  amplifier  is  fabricated  using  the 
TSMC  0.35-pm  two-poly  four-metal  complementary  metal-oxide  semiconductor  (CMOS) 
process  and  mounted  on  a  dedicated  printed-circuit  board  for  testing.  To  measure  the  chip 
performance,  a  DAQ  device  (National  Instruments,  USA)  is  used  to  collect  and  transmit  the  16- 
channel  signal  to  a  laptop.  The  DAQ  device  also  provides  a  200-kHz  clock  to  operate  the  MUX. 
A  microphotograph  of  the  complete  chip  is  shown  in  figure  9.  The  whole  chip  achieves  a  size  of 
4.18  mm“  including  pads.  It  drains  a  power  of  108  pW  when  operating  for  full  16-channel  neural 
recording.  The  chip  operates  from  a  single  1.5-V  supply  without  off-chip  components.  The 
measured  input  referred  noise  is  2  pVrms  in  the  band  of  DC  to  200  Hz. 
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Figure  9.  A  microphotograph  of  the  complete  16-channel  neural  amplifier  chip. 

Testing.  We  have  conducted  a  comparison  study  between  the  physiological  signals  (simulated 
electrocardiogram  [ECG])  obtained  by  our  custom  designed  and  fabricated  low-noise  AFE 
(DDA)  and  a  commercially  available  1 -channel  DDA  (LT1789)  from  Linear  Technology  (figure 
10).  The  correlation  between  the  two  amplified  signals  was  close  to  1. 
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Furthermore,  to  verify  the  neural  recording  capability  of  the  present  AFE  chip,  simultaneous 
recording  performance  comparison  with  commercial  integrated  circuits  (ICs)  is  tested  by  using  a 
neural  signal  simulator  (Cyberkinetics,  US).  A  similar  three-stage  amplifier  structure,  which 
consisted  of  an  instrumentation  amplifier  LT1789  (Linear  Technology,  U.S.)  and  two  operational 
amplifiers  AD8607  (ADI,  U.S.),  was  used  for  comparison.  Both  present  AFE  chip  and 
commercial  ICs  provides  a  band  of  0.2-6  kHz  with  66  dB  in  gain.  A  100-ms  segment  of 
comparison  result  is  shown  in  figure  11.  In  figure  11,  the  upper  solid  line  and  the  lower  dotted 
line  display  the  results  from  present  neural  amplifier  and  commercial  ICs,  respectively.  These 
two  recordings  are  highly  correlated  on  average  (r  >0.99),  demonstrating  the  usefulness  of  the 
present  chip  for  neural  recording. 


Figure  11.  A  comparison  between  our  neural  amplifier  and  a  commercial  IC  using  signals 
generated  from  a  neural  signal  simulator.  The  solid  line  and  dotted  lines  show 
the  recordings  from  our  AFE  amplifier  and  the  commercial  IC,  respectively. 

Table  1  summarizes  the  comparison  of  the  measured  parameters  of  this  AFE  chip  with  those  of 
reported  works.  It  can  be  seen  that  present  neural  amplifier  offers  technical  merits  of  reduced 
supply  voltage,  sufficient  low  power  per  channel,  and  reasonable  low-noise  performance,  yet 
offers  comparable  measured  results  such  as  input  offset,  CMRR  and  power  supply  rejection  ratio 
(PSRR). 
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Table  1.  A  comparison  between  our  neural  amplifier  and  a  commercial  IC  using  signals  generated  from  a  neural 
signal  simulator.  The  solid  line  and  dotted  lines  show  the  recordings  from  our  AFE  amplifier  and  the 
commercial  IC,  respectively. 


This  work 

Harrison  & 
Charles,  2003 

Harrison, 

2007 

Aziz  et  al„ 
2009 

Mollazadeh  et 
al.,  2009 

Technology 

TSMC  0.35  pm 

AMI  1.5  pm 

AMI  0.6  pm 

CMC  0.35  pm 

AMI  0.5  pm 

Simulation 

Measurement 

Measurement 

Measurement 

Measurement 

Measurement 

Supply  voltage  (V) 

1.5 

1.5 

±2.5 

5 

3 

3.3 

Current  per 
channel  (pA) 

4 

4.5 

16 

35 

5 

12 

Power  dissipation 
per  channel  (gW) 

6.1 

6.75 

13.33 

175 

15 

39.56 

Number  of 
channels 

16 

16 

6 

16 

256 

16 

Chip  area  (mm2) 

4.18 

0.16 

13.33 

15.75 

9 

Mid-band  gain 
(dB) 

52-75 

48-65 

39.5 

46 

48-68 

39.6 

High-pass 
frequency  (Hz) 

0.1 

<0.2 

0.025 

0.05 

0.01-70 

0.2-94 

Low-pass 
frequency  (Hz) 

50-1 Ok 

50-6k 

7.2k 

10-10k 

5 00-5 k 

140-8.2 

Input  offset 
voltage  (V ) 

lOOn 

87. 5u 

— 

±0.6m 

— 

— 

Input  common 
mode  range(  V ) 

0.2-1. 3 

0.26-1.28 

— 

— 

— 

— 

Input-referred 

(pVrms) 

0.944 

2 

2.2 

2 

7 

1.94 

PSRR  (dB) 

120 

>78 

>85 

75 

- 

70 

CMRR  (dB) 

158 

>90 

>83 

84 

- 

76 

1.5  Conclusions 

In  the  past  year,  our  research  team  has  successfully  designed  and  developed  dry  EEG  and  fNIR 
sensors  that  allow  non-invasive  and  non-intrusive  acquisition  of  EEG  and  fNIR  signals.  After 
numerous  simulations  and  tests,  we  have  finalized  the  design  of  the  form  factor  of  electro-optode 
that  integrates  the  EEG  and  fNIR  sensors.  We  have  also  fabricated  some  samples  for  testing. 

1.6  Planned  Activities  During  the  Next  Report  Period 

In  the  next  period,  we  plan  to  accomplish  the  following: 

1.  Conduct  a  series  of  in-vitro  experiments  on  pigskin  to  test  the  quality  of  NIR  signals 
acquired  by  the  newly  fabricated  electro-optodes. 

2.  Conduct  pilot  experiments  on  human  fingers/arms  and  scalp  to  test  the  stability  of  signals 
acquired  by  the  electro-optodes,  the  16-channel  AFE  very  large-scale  integration  (VLSI) 
chip  and  the  supporting  PCB  under  different  experimental  conditions.  The  goal  of  this 
study  is  to  evaluate  the  penetration  rate  and  efficiency  of  the  NIR  lights  of  the  electro- 
optodes. 
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2.  Project  2:  EEG  Pattern  Classification  Using  QStates 


2.1  Authors 

The  authors  of  this  project  are  Anthony  J.  Ries  and  Kaleb  McDowell  of  ARL. 


2.2  Introduction/Objective 

Using  patterns  of  brain  activity  as  indicators  of  a  particular  cognitive  state  offers  the  potential  to 
monitor  and  enhance  Soldier  performance.  For  example,  monitoring  brain  signals  related  to 
Soldier  fatigue  would  assist  in  determining  if  the  Soldier  is  alert  enough  to  perform  certain 
cognitively  demanding  tasks  such  as  operating  unmanned  aerial  vehicles  (UAVs).  EEG  pattern 
classification  has  been  used  to  discriminate  between  a  number  of  cognitive  processes.  Flowever, 
before  this  technology  is  implemented  on  the  battlefield  and  decisions  are  made  based  on  brain 
state  classification,  it  must  be  rigorously  tested  and  validated.  We  recently  acquired  QStates,  a 
software  suite  transitioned  from  the  U.S.  Army  Research  Office  (ARO),  which  processes  and 
classifies  patterns  of  EEG.  This  report  summarizes  our  initial  findings  on  QStates  utility  in 
classifying  electrical  brain  activity  recorded  from  two  simulated  missions. 


Figure  12  shows  the  general  process  of  extracting  and  applying  information  from  patterns  of 
brain  activity  collected  with  EEG  to  train  classification  models  and  classify  new  data  using  those 
models.  The  raw  EEG  signal  can  be  processed  in  real  time  or  offline.  The  preprocessing  step 
removes  signal  artifacts  such  as  eyeblinks  and  high  frequency  muscle  activity,  re -references  the 
data,  extracts  specific  signal  features  such  as  power  spectral  density  (PSD),  and  segments  the 
data  into  epochs.  Feature  extraction  most  often  entails  a  linear  transformation  of  the  preprocessed 
epochs  to  discover  patterns  that  best  differentiate  the  states  of  interest  (e.g.,  state  X  versus  state 
Y),  while  feature  selection  reduces  the  feature  dimensionality  while  still  trying  to  obtain  high 
classification  performance.  The  feature  extraction  and  selection  process  produces  a  feature  vector 
for  each  state  that  is  used  to  train  a  classifier  or  is  used  as  input  to  classify  new  data.  The 
classification  output  indicates  which  state  the  data  likely  represent  at  given  timepoints  according 
to  the  trained  model. 


Figure  12.  Flow  chart  depicting  the  basic  process  of  acquiring,  processing,  and  classifying  EEG  data. 
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QStates  is  a  software  package  for  the  offline  analysis  of  EEG  and  other  physiological  data.  It  has 
been  designed  to  process  EEG  data  collected  during  user-specified  cognitive  tasks,  extract 
psychophysiological  features  from  the  data  relevant  to  a  subject’s  state,  and  classify  patterns  of 
brain  activity  using  either  a  user-designed  classification  model  or  one  of  three  pre-installed 
normative  models  developed  by  the  manufacturer.  The  pre-installed  models  claim  to  assess  the 
following  cognitive  states:  Engagement,  Workload,  and  Fatigue.  Cognitive  Engagement, 
Workload,  and  Fatigue  are  characterized  by  the  following: 

•  Engagement:  increase  in  Q  power,  decrease  in  a  power 

•  Workload:  increase  in  Q  power,  decrease  in  a  power  (though  the  magnitudes  of  the 
changes  are  typically  to  those  observed  for  Engagement) 

•  Fatigue:  increase  in  6  power,  decrease  in  p  power 

The  partial  least  squares  (PLS)  algorithm  uses  features  extracted  from  the  physiological  data  and 
the  ground  truths  in  table  2  as  inputs  to  create  the  three  normative  models.  A  ground  truth  of  zero 
corresponds  to  the  “Low  state”  for  the  classifier,  whereas  a  ground  truth  of  1  corresponds  to  the 
“High  state.”  The  Relaxed  state  is  not  used  when  training  a  classifier  for  Workload  and  was 
therefore  assigned  an  X  for  its  ground  truth. 

Table  2.  Tasks  and  ground  truth  assumptions  used  to  create  the  normative  models 
delivered  with  the  QStates  classification  software. 


Task 

Engagement 

Workload 

Relaxed.  Eyes  Open 

0 

X 

Column  Add  (1  digit) 

1 

0 

Column  Add  (3  digit) 

1 

1 

BattleField2  (no  ememes) 

1 

0 

BattleField2  (5  or  6  ememes) 

1 

1 

Many  classification  software  packages  contain  preinstalled  cognitive  state  models  designed  from 
a  normative  dataset;  however,  it  is  not  clear  how  well  these  models  perform  when  classifying 
cognitive  states  from  subjects  and  tasks  different  from  the  ones  from  which  they  were  created. 
For  example,  the  Qstates  software  contains  a  normative  workload  model  that  is  presumably 
sensitive  to  cognitive  states  associated  with  low  and  high  workload.  This  model  was  created 
using  data  from  a  first-person  shooter  game  and  a  math  task  where  the  low  workload  classifier 
was  trained  on  cognitive  processes  generated  from  a  zero-enemy  shooting  condition  and  one¬ 
digit  addition  math  task,  while  the  high  workload  classifier  was  trained  cognitive  processes 
generated  from  a  six-enemy  shooting  condition  and  three-digit  addition  math  task.  Alternatively, 
the  software  allows  the  user  to  construct  their  own  models  using  EEG  data  from  any  task  to 
classify  on  any  number  of  states.  This  also  makes  it  possible  to  classify  data  from  the  same 
subjects  but  from  different  from  those  used  to  develop  the  classification  model.  Using  models 
that  generalize  to  other  tasks  different  from  those  they  were  trained  on  assumes,  of  course,  that 
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the  states  in  the  training  and  testing  data  are  similar.  In  the  present  study  we  compared  the 
classification  accuracy  of  QStates  generalized  workload  model  with  two  user-created  models 
using  data  obtained  from  a  driving  simulation  (Experiment  1)  as  well  as  an  interactive  first- 
person  shooter  exercise  (Experiment  2). 

2.3  Experiment  1 

2.3.1  Approach:  Experiment  1 

For  Experiment  1,  we  compared  the  QStates  normative  workload  model  with  task- specific 
individual  and  group  models.  Informed  consent  was  obtained  prior  to  experimentation  as 
required  by  U.S.  Army  human  use  regulations  (U.S.  Department  of  Defense,  1999;  U.S. 
Department  of  the  Army,  1990).  Soldiers  performed  a  visual  scanning  task  under  two  indirect 
driving  conditions  on  a  ride  motion  simulator  (RMS)  while  EEG  was  recorded  from  a  64-channel 
BioSemi  Active  Two  system.  Soldiers  performed  two  scanning  tasks  in  each  condition.  In  the 
first  condition,  coupled  motion,  there  was  a  direct  match  between  the  virtual  and  physical 
environments.  For  example,  a  Soldier  would  feel  the  acceleration  of  the  vehicle  on  the  RMS  the 
instant  the  Soldier  saw  it  move  forward  in  the  simulation.  In  the  other  condition,  uncoupled 
motion,  what  the  Soldier  felt  on  the  RMS  was  delayed  11s  from  what  the  Soldier  saw  in  the 
simulation.  It  was  believed  that  the  uncoupled  motion  condition  would  generate  distinct  brain 
activation  patterns  from  the  coupled  motion  condition  due  to  the  mismatch  between  physical 
feedback  and  visual  input. 

The  Individual  model  used  one  of  the  two  tasks  within  each  condition  to  train  the  model  and 
performed  classification  on  the  other.  This  was  done  separately  for  each  individual;  therefore, 
each  participant  had  his/her  own  model.  The  task  order  was  counterbalanced  for  training  and 
classification  purposes.  The  Group  model  used  all  subjects’  data  simultaneously  to  train  model. 
The  Normalized  Workload  model  came  preinstalled  with  the  QStates  software  and  used  a 
different  sample  of  subjects  who  performed  the  tasks  outlined  above  in  table  2.  We  refer  to  the 
coupled  motion  condition  (Individual  and  Group  models)  and  one-enemy/one-digit  addition 
(Normative  Workload  model)  as  “Low”  processing  state  and  the  uncoupled  motion  condition 
(Individual  and  Group  models)  and  six-enemy/three-digit  addition  (Normative  Workload  model) 
as  “High”  processing  state. 

The  QStates  software  outputs  a  score  indicating  the  probability  a  given  two-second  data  epoch 
belongs  to  the  state  trained  as  “High”  in  the  model.  In  other  words  if  the  probability  of  a  given 
data  epoch  is  above  0.5,  then  it  is  more  similar  to  the  data  used  to  train  the  “High”  state  portion 
of  the  model.  If  the  output  is  below  0.5,  the  data  epoch  is  more  similar  to  the  data  used  to  train 
the  “Low”  state  portion  of  the  model.  Scores  nearest  to  0.5  indicate  the  data  epoch  is  unable  to  be 
accurately  classified  according  to  the  model. 
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2.3.3  Results:  Experiment  1 


To  assess  the  ability  of  the  models  to  discriminate  between  two  condition-dependent  states 
(coupled  motion/low;  uncoupled  motion/high),  we  compared  the  average  probability  output  for 
each  condition  using  each  model  (figure  13).  Outputs  for  coupled  and  uncoupled  motion 
conditions  in  the  Individual  and  Norm  Workload  models  were  not  statistically  different  from 
chance  classification  (all  p  >  0.1).  Both  conditions  in  the  Group  model  were  significantly 
different  from  chance  classification,  ([  Coupled  t(9)  =  -4.4,  p  =  0.002];  [Uncoupled  t(9)  =  -2.4, 
p  =  0.04]).  However,  coupled  and  uncoupled  motion  output  was  not  significantly  different  from 
each  other,  (t(9)  =  —1.9,  p  =  0.08)  and  the  uncoupled  motion  condition,  while  different  from 
chance,  was  classified  as  more  likely  to  come  from  the  same  state  produced  in  the  coupled 
motion  condition  (i.e.,  Low). 


Figure  13.  Classification  results  for  the  coupled  and  uncoupled  motion  conditions  using  the  Qstates 
Normative  Workload  model  and  Individual  and  Group  models.  Error  bars  represent  95% 
confidence  intervals. 

2.3.3  Conclusion:  Experiment  1 

The  analysis  of  the  classification  data  in  Experiment  1  suggests  the  main  manipulation  in  the 
experiment  was  not  effective,  for  there  were  no  significant  classification  differences  between  the 
two  conditions  for  any  model.  This  corroborates  the  behavioral  findings  that  showed  no 
statistically  significant  difference  in  reaction  time  and  accuracy  between  the  two  conditions. 
Given  the  results  from  Experiment  1  analysis,  we  analyzed  a  different  dataset  that  did  show  an 
effective  experimental  manipulation  as  revealed  by  significantly  different  reaction  time  and 
accuracy  scores  in  the  conditions  of  interest.  Given  there  were  differences  in  performance,  we 
expected  to  find  these  differences  reflected  in  the  neural  data. 
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2.4  Experiment  2 

2.4.1  Approach:  Experiment  2 

In  Experiment  2,  we  compared  the  pre-installed  Normative  Workload  model  with  Individual  and 
Group  models  generated  from  14  marksmen  (13  Marines,  1  Soldier).  Marksmen  performed  four 
separate  tasks  while  EEG  was  recorded  from  a  40-channel  neuroscan  system.  The  first  task,  eyes 
open  (EO),  required  each  individual  to  simply  relax  with  their  eyes  open.  For  the  second  task, 
subjects  performed  two-digit  math  problems,  the  math  (M)  condition.  In  Task  3,  shooting  (S), 
each  subject  performed  a  first-person  shooting  task  in  the  Dismounted  Infantryman  Survivability 
and  Lethality  Testbed  (DISALT)  shooting  simulator.  In  this  task  the  marksmen  fired  at  targets 
from  50-300  m.  In  the  last  task,  shooting  and  math  (S&M),  the  shooters  performed  both  the 
shooting  and  math  tasks  together. 

The  Individual  model  used  only  the  individual  subject’s  M  and  EO  data  to  train  model.  The 
Group  model  used  all  14  subjects’  M  and  EO  data  to  train  model.  The  Normalized  Workload 
came  preinstalled  with  the  QStates  software  and  used  a  different  sample  of  subjects  who 
performed  the  tasks  outlined  previously.  We  refer  to  the  EO  class  (Individual  and  Group  models) 
and  one-enemy/one-digit  addition  (Normative  Workload  model)  as  “Low”  processing  state  and 
M  task  class  (Individual  and  Group  models)  and  six-enemy/three-digit  addition  (Normative 
Workload  model)  as  “High”  processing  state. 

2.4.2  Results:  Experiment  2 

The  first  analysis  tested  to  see  if  there  was  a  difference  between  the  S  and  S&M  classification 
output  for  each  model.  Based  on  subjective  feedback  it  was  suggested  that  the  S&M  task 
required  more  mental  effort  than  the  S  task  alone;  therefore,  we  believed  the  S&M  task 
classification  output  would  be  more  probable  than  the  S  task  alone  to  be  classified  in  the  high 
processing  state.  Figures  14  and  15  show  the  output  for  the  S  and  S&M  tasks,  respectively,  for 
each  model  across  epochs  and  Figure  16  shows  the  means  and  95%  confidence  intervals. 

Overall,  there  was  no  significant  difference  between  S  and  S&M  classification  output  for  the 
Individual  (t  =  0.328,  p  =  0.748),  Group  ( t  =  -0.007,  p  =  0.994),  and  Normative  Workload  (t  = 
1.866,  p  =  0.085).  While  the  Normative  Workload  model  approached  significance,  the  output 
suggested  the  S  task  was  more  difficult  than  the  S&M  task,  which  is  counter  to  the  subjective 
task  difficulty  reported  by  the  participants. 

The  second  analysis  tested  the  classification  accuracy  for  each  model  by  comparing  the  task 
output  to  a  probability  of  0.5.  This  analysis  revealed  that  only  the  Individual  and  Group  models 
produced  classification  significantly  above  chance  level  for  both  tasks  (all  p  <  0.01).  The  output 
for  the  S  and  S&M  tasks  was  not  different  from  chance  classification  using  the  Normative 
Workload  model  (p  >  0.1). 
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It  is  important  to  note  that  while  the  training  tasks  in  all  models  were  similar,  the  Normative 
Workload  model  training  tasks  may  have  been  more  difficult  or  produced  greater  mental  effort  in 
the  subject  sample.  If  the  S  or  S&M  task  required  the  most  mental  effort  in  the 
testing/classification  data  but  required  less  mental  effort  when  compared  to  the  high  mental 
effort  task  in  the  Normative  Workload  model,  then  it  is  not  surprising  that  the  S  and  S&M  output 
had  a  lower  probability  of  being  in  the  high  state  when  using  the  Normative  Workload  model.  In 
other  words,  the  classification  output  is  dependent  upon  the  training  data  used  in  the  model.  If 
the  training  data  between  two  models  are  different  and  reflect  different  degrees  of  cognitive 
processing  initially,  then  it  would  not  be  surprising  to  get  different  classification  output  as  well. 
This  is  a  key  concern  when  employing  normative  models  especially  when  the  specifics  of  the 
training  data  are  unknown. 


Figure  14.  Classifier  output  over  the  course  of  the  S  task  for  a  single  subject. 
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Figure  15.  Classifier  output  over  the  course  of  the  S&M  task  for  a  single  subject. 


Figure  16.  Classification  results  for  the  S  and  S&M  tasks  using  the  Qstates  Normative  Workload  model  and 
individual  and  group  models  trained  on  the  EO  and  M  tasks.  Error  bars  represent  95%  confidence 
intervals. 

Workload  related  brain  activity  is  sensitive  to  changes  in  mental  effort  devoted  to  a  particular 
task.  In  the  current  tasks,  it  is  likely  that  the  S&M  task  required  more  mental  effort  than  the  S 
task  alone;  however,  the  classification  output  from  the  Normative  Workload  model  suggests  the 
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opposite  is  true.  This  brings  into  question  the  external  validity  of  the  Normative  Workload  model 
and  reinforces  the  idea  that  normative  models  of  workload  are  not  always  consistent  with  the 
ground  truth. 

4.2.3  Conclusions:  Experiment  2 

Due  to  the  high  variability  in  brain  activity  between  individuals,  it  is  recommended  that 
Individual  models  are  used.  Group  and  Normative  models  generally  produce  lower  classification 
accuracy  compared  to  Individual  models,  and  therefore,  output  based  on  Group  and  Normative 
models  should  be  interpreted  with  caution.  Moreover,  it  is  essential  that  the  researcher  know 
what  tasks  were  used  in  creating  normative  pre-installed  models,  for  not  all  tasks  require 
cognitive  processes  that  are  easily  generalizable. 

Our  suggestions  for  more  enhanced  classification  software  are  as  follows: 

•  Enable  the  user  to  perform  event-related  classification. 

•  Offer  more  algorithm  selections  such  as  Support  Vector  Machines  (SVM). 

•  Increase  number  of  subjects  and  tasks  used  to  create  the  normative  QStates  models. 

•  Incorporate  temporal  aspects  of  cognitive  processes  into  possible  classification  parameters. 


3.  Project  3:  Non-linear  Brain  Activity  in  Real-world  Settings:  Movement 
Artifact  and  the  Phase  Lag  Index 


3.1  Authors 

The  authors  of  this  project  are  Kaleb  McDowell,  Scott  E.  Kerick,  and  Kelvin  Oie. 

3.2  Introduction/Objective 

The  brain  is  a  complicated  network  comprised  of  neural  systems  that  function  to  meet  two 
diverse  requirements:  the  local  specialization  of  assemblies  of  neurons  for  specific  types  of 
information  processing  and  the  integration  of  information  at  a  more  global  level  (Friston,  1994; 
Spoms,  Tononi,  and  Edelman,  2000).  Recently,  it  has  been  argued  that,  in  order  to  advance  our 
current  understanding  of  functional  brain  dynamics,  a  more  large-scale,  integrative  systems  view 
is  required  (Le  van  Quyen,  2003;  Varela,  Lachaux,  Rodriguea,  and  Martinerie,  2001).  Recently, 
novel  approaches  have  started  to  be  implemented  to  examine  the  nervous  system  at  this 
integrative  level.  For  example,  the  rise  of  network  science  (Albert  and  Barabasi,  2002;  Borner, 
Sanyal,  and  Vespignani,  2007;  Newman,  2003;  Strogatz,  2001;  Watts  and  Strogatz,  1998)  has 
contributed  novel  approaches  that  are  now  being  applied  to  examine  brain  function  at  an 
integrated  global  level  (de  Haan  et  al.,  2009;  De  Vico  Fallani  et  al.,  2007;  Reijneveld,  Ponten, 
Berendse,  and  Stam,  2007;  Stam,  2007,  2010). 
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One  of  the  critical  components  of  any  approach  underlying  an  integrative  systems  view  will  be  to 
understand  the  connectivity  between  local  regions  within  the  nervous  system.  Traditional 
techniques  to  examine  this  connectivity,  such  as  cross-correlation  and  coherence,  are  based  on 
linear  techniques.  However,  it  has  been  posited  that  coordinated  neural  activity  may  be  related 
through  nonlinear  functions,  including  those  with  strong  transients  or  cross-frequency  phase 
locking  (Friston,  2000;  Palva,  Palva,  and  Kaila,  2005;  Stam,  Breakspear,  van  Cappellen  van 
Walsum,  and  van  Dijk,  2003),  and  traditional  linear  techniques  are  limited  by  their  inability  to 
detect  nonlinear  interdependencies  between  the  underlying  dynamical  systems,  their  unsuitability 
to  characterize  non-stationary  data  with  rapidly  changing  interdependencies,  and  the  potential 
bias  for  high  correlations  among  EEG  recording  electrodes  due  to  volume  conductive  properties 
of  the  cerebral- spinal  fluid,  meninges,  skull,  and  scalp  (Stam  et  al.,  2009;  Stam  and  van  Dijk, 
2002).  One  new  nonlinear  measure  of  functional  connectivity  that  purportedly  circumvents  these 
issues  is  the  phase  lag  index  (PLI)  (Stam,  Nolte,  and  Daffertshofer,  2007).  PLI  is  a  measure  of 
the  asymmetry  of  the  distribution  of  phase  differences  between  two  signals  and  is  less  affected 
by  volume  conduction  than  traditional  linear  measures  such  as  coherence  and  nonlinear  measures 
such  as  synchronization  likelihood.  This  is  because  PLI  assumes  that  the  phase  lag  of  volume- 
conducted  signals  is  near-instantaneous  (i.e.,  zero  phase)  (Stam  et  al.,  2007;  Stam  et  al.,  2009) 
and,  therefore,  non-zero  phase  relationships  observed  between  two  interdependent  signals  cannot 
be  attributed  to  volume  conduction,  but  may  be  attributed  to  a  functional  relationship  between 
them. 

Thus,  although  promising,  PLI  has  only  been  applied  to  small  segments  of  artifact-free  EEG 
data,  which  potentially  limits  its  application,  especially  with  regards  to  understanding  brain 
networks  of  Soldiers  performing  tasks  in  operational  environments  where  artifacts  are 
unavoidable.  Accordingly,  the  purpose  of  this  study  was  to  apply  PLI  analyses  to  EEG  data 
recorded  from  participants  under  conditions  of  varying  levels  of  motion  characteristic  of 
operational  environments.  Specifically,  we  subjected  participants  to  stationary  (sitting  still)  and 
dynamic  vehicle  motion  simulation  conditions  (sitting  in  a  seat  while  traversing  over  a  paved 
“washboard”  road  surface  and  traversing  over  irregular  “cross-country”  terrain)  and  stationary 
(standing  still)  and  dynamic  ambulatory  conditions  (walking  and  jogging  on  a  treadmill). 

3.3  Approach 

3.3.1  Subjects 

The  test  participants  were  normal  adult  right-hand,  right-eye-dominant  males  (N  =  5;  age  range 
27-39).  The  voluntary,  fully  informed  consent  of  the  persons  used  in  this  research  was  obtained 
as  required  by  U.S.  Army  human  use  regulations  (U.S.  Department  of  Defense,  1999;  U.S. 
Department  of  the  Army,  1990). 
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3.3.2  Motion  Environments 

A  RMS  was  used  to  simulate  vehicle  motion  environments.  The  RMS  uses  a  MOOG  6  degree- 
of-freedom  (6-DOF)  20000E  motion  platform  (MOOG,  East  Aurora,  NY),  capable  of  producing 
dynamics  similar  to  that  of  military  ground  vehicles  traversing  over  secondary  roads  and  cross¬ 
country  terrain.  It  is  comprised  of  a  platform  mounted  on  a  hexapod  actuator  design  that  is 
securely  fixed  to  a  non-movable  surface  and  produces  motions  in  the  longitudinal,  lateral, 
vertical,  roll,  pitch,  and  yaw  directions  with  maximal  acceleration  limited  to  ±0.6  g  lateral  and 
longitudinal  and  -0.5  to  0.7  g  vertical.  A  treadmill  (Quinton,  Bothell,  WA)  was  used  to  simulate 
ambulatory  motion  environments. 

3.3.3  Electroencephalography 

Continuous  EEG  data  were  acquired  using  a  24-bit,  40-channel  ActiveTwo  amplifier  with 
ActiView  software  and  an  electrode  cap  (Active  Headcap)  with  pre-amplified  surface  electrodes. 
A  water-soluble  electrode  gel  (Signa  Gel;  Parker  Laboratories,  Inc.,  Fairfield,  NJ)  was  inserted 
into  each  of  the  electrode  casings  to  facilitate  conductivity  between  the  scalp  and  electrode 
surfaces.  EEG  data  were  recorded  from  32  electrode  sites  referenced  to  a  common  mode  sense 
(CMS)  electrode  with  the  midline  frontopolar  electrode  (FPz)  as  ground.  Data  were  re-referenced 
offline  to  averaged  mastoids  (Al,  A2).  Electrode  impedances  were  maintained  under  10  k£2.  The 
sampling  rate  for  analog-to-digital  conversion  was  512  Hz  and  band  pass  filtered  at 
0.016-100  Hz  online.  Vertical  (VEOG)  and  horizontal  (HEOG)  electro-ocular  activity  was 
recorded  using  bipolar  montages  attached  superior  and  inferior  to  the  right  eye  and  both  orbital 
fossa. 

3.3.4  Task  and  Conditions 

The  experiment  consisted  of  an  auditory  discrimination  task  using  the  “oddball”  paradigm 
(Polich  and  Kok,  1995;  Polich,  1997),  which  the  participants  performed  in  each  of  six  motion 
environment  conditions.  In  each  motion  environment  condition,  a  total  of  100  auditory  stimuli 
(target  and  non-target  stimuli)  were  presented  once  every  3. 5-6. 5  s  in  a  random  series  with  the 
target  stimuli  (2000  Hz)  occurring  less  frequently  (probability  =  0.20)  than  the  non-target 
(1000  Hz;  probability  =  0.80)  stimuli.  All  auditory  stimuli  were  50  ms  in  duration  and  presented 
at  60  dB.  Inter- stimulus  intervals  were  varied  to  reduce  the  likelihood  of  habituation  via 
expectancy  effects  (Yordanova  and  Kolev,  1997).  The  auditory  stimuli  were  generated  by  a 
personal  computer  running  E- Prime  (ver.  1.1;  Psychology  Software  Tools,  Inc.,  Pittsburgh,  PA) 
and  presented  binaurally  to  the  participants  via  stereo  headphones,  which  were  worn  throughout 
all  task  conditions.  The  participants  held  a  modified  bicycle  grip  (analogous  to  a  gaming 
joystick)  in  their  right  hand  and  were  instructed  to  press  a  button,  which  was  attached  to  the  top 
of  the  grip,  with  their  thumb  as  quickly  as  possible  in  response  to  target  stimuli  and  to  not 
respond  to  non-target  stimuli.  Participants  were  instructed  to  fixate  on  a  spot  marked  directly  in 
front  of  them  at  eye-level.  Response  times  were  recorded  as  the  time  interval  between  the  onset 
of  target  stimuli  and  the  time  of  each  relevant  button  press.  Response  errors  (i.e.,  the 


22 


combination  of  percent  failures  to  respond  to  target  stimuli  and  the  percent  of  responses  to  non¬ 
target  stimuli)  were  also  recorded. 

The  six  motion  environment  conditions  included  three  vehicle  motion  environments  and  three 
ambulatory  motion  environments.  For  the  vehicle  motion  environments,  a  “control”  condition 
required  the  participants  to  perform  the  auditory  discrimination  task  from  a  static  seated  position 
in  a  chair  mounted  on  the  RMS  platform  with  no  motion.  In  a  low  vehicle  motion  condition,  the 
participants  performed  the  task  while  riding  on  the  RMS  simulating  a  wheeled  vehicle  traversing 
over  a  paved  washboard  road  surface  and  in  a  high  motion  condition  simulating  a  tracked  vehicle 
traversing  cross-country  terrain.  For  the  ambulatory  motion  environments,  the  participants 
performed  the  auditory  discrimination  task  while  standing  still  on  a  treadmill  with  no  motion 
(0  mph),  while  walking  on  a  treadmill  (3  mph),  and  while  jogging  on  a  treadmill  (5  mph)  (for 
more  details  see  Kerick,  McDowell,  and  Oie,  2009).  A  2-min  warm-up  was  provided  for  the 
jogging  condition  to  minimize  risk  of  injury  by  gradually  increasing  the  treadmill  up  to  speed. 
The  sequence  of  conditions  was  assigned  randomly  within  blocks  of  RMS  (control,  paved,  and 
cross-country)  and  treadmill  (standing,  walking,  and  jogging)  environments  and  blocks  of  RMS 
and  treadmill  conditions  were  counterbalanced  to  minimize  order  effects.  Each  task  condition 
was  ~6  min  in  duration,  with  5-min  rest  provided  between  each  task  performance.  The  total  time 
to  complete  the  experimental  session  was  90-120  min. 

3.4  Results 

In  a  previous  effort  (Kerick,  Oie,  and  McDowell,  2009),  we  investigated  whether  we  could 
derive  reliable  event -related  potentials  (ERPs)  in  each  of  the  environments  and  found  that  ERPs 
could  not  be  reliably  obtained  in  the  cross-country  vehicle  motion  and  walking  and  jogging 
ambulatory  conditions  using  standard  methods  because  of  excessive  artifacts  observed  in  these 
conditions.  However,  that  effort  revealed  no  differences  in  behavioral  measures  in  any  of  the  six 
conditions. 

Here,  we  attempt  to  understand  the  susceptibility  of  PLI  to  the  motion  artifacts  generated  in  these 
six  environments.  However,  due  to  the  novel  nature  of  the  PLI  measure,  the  PLI  methods  will  be 
intertwined  with  the  results. 

To  give  a  general  concept  of  the  artifacts  in  this  study,  figure  17  is  presented  which  is  a  sample 
of  10  s  of  “nearly”  raw  data  from  each  of  the  six  conditions.  Generally,  the  raw  data  from  the 
sitting,  paved,  and  standing  conditions  are  similar;  however,  artifacts  can  be  observed  directly  in 
the  other  three  conditions  including  the  stride  frequency  being  clearly  evident  in  the  jogging 
condition  (see  Kerick  et  al.,  2009,  for  details). 
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Figure  17.  EEG  data  at  Cz  from  a  single  participant  after 

0.01-40  Hz  band  pass  filtering  and  eye-blink  reduction. 

Each  line  represents  a  continuous  10-s  segment  of 
data  from  within  each  environment  (figure  from 
Kerick  et  ah,  2009).  Note  that  eye -blink  reduction. 

Consistent  with  Stam  et  al.  (2007),  the  raw  EEG  data  were  first  filtered  into  six  standard  EEG 
frequency  bands  (delta  [0.5-4  Hz],  theta  [4-8  Hz],  alpha  1  [8-10  Hz],  alpha  2  [10-13  Hz],  beta 
[13-30  Hz],  and  gamma  [30-45  Hz])  using  a  finite  impulse  response  filter.  PLI  was  then 
calculated  using  a  Hilbert  transform  and  formulas  from  Stam  et  al.  (2007).  Underlying  each 
estimate  of  PLI  is  a  phase  estimate  over  a  given  reference  (RF)  window  length.  To  examine  the 
effect  of  RF  on  PLI,  PLIs  for  a  single  electrode  pair  were  continually  calculated  for  multiple  RFs 
from  250  ms  to  8  s  in  250-ms  intervals.  For  each  window  length,  PLIs  were  estimated  multiple 
times  within  a  16-s  epoch  by  stepping  through  the  epoch  at  250-ms  increments  resulting  in  63 
PLI  estimates  for  the  smallest  RF  and  33  estimates  for  the  longest  RF. 

As  can  be  seen  in  figure  18,  as  RF  changes,  the  variability  of  the  PLI  estimate  can  change, 
further  as  RF  increases,  PLI  decreases. 
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Figure  18.  PLI  estimates  at  a  single  electrode  pair  from  a 
single  participant.  High  variability  and  high 
amplitude  is  observed  in  PLIs  calculated  with 
small  (250-ms)  RF. 

For  further  analyses,  PLI  was  calculated  using  an  RF  of  500  ms  in  250-ms  intervals  for  each 
electrode  pair  and  for  each  frequency  bin.  PLIs  were  calculated  within  16-s  epochs  centered  on 
each  stimulus  presentation  resulting  in  61  PLI  estimates  for  each  stimulus  to  attempt  to 
understand  the  susceptibility  of  PLI  to  the  motion  artifacts  generated  in  the  six  environments. 
Here,  mean  PLI  were  calculated  across  all  participants,  all  electrode  pairs,  time,  and  both 
stimulus  types.  Figure  19  illustrates  a  normalized  version  of  these  means  for  each  condition  and 
for  each  frequency  band.  Normalization  was  achieved  by  subtracting  the  mean  from  the  sitting 
condition  from  that  for  each  condition;  as  such,  the  normalized  PLIs  for  the  sitting  condition  are 
zero.  Further,  there  are  several  options  for  representing  the  variance  of  these  data  (e.g.,  across 
participants,  electrode  pairs,  time).  Here,  we  choose  to  present  the  standard  deviation  across  the 
16-s  epochs  prior  to  averaging  across  participants  and  electrode  pairs  to  reflect  how  the  measure 
generally  varies  on  a  moment  to  moment  basis  (between  participants  variance  is  presented  in 
figure  20).  While  it  is  clear  that  we  cannot  definitively  differentiate  between  the  susceptibility  of 
PLI  to  motion  artifact  versus  differences  in  neural  process  in  these  conditions,  we  believe  that 
this  illustration  effectively  describes  the  generic  behavior  of  the  measure  across  the  six 
environments  examined  here. 
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Figure  19.  Normalized  PLI  estimates  for  all  participants, 
all  electrode  pairs,  and  both  stimulus  types. 
Within  each  condition,  frequencies  from  left  to 
right  are  ordered  from  delta  to  gamma  band 
(low  to  high).  Bars  reflect  the  mean  standard 
deviations  across  epochs. 


Figure  20.  Event-related  PLI  estimates  for  all  participants, 
all  electrode  pairs,  and  the  target  stimuli. 

Stimulus  onset  occur  here  at  time  =  0,  however, 
due  to  the  RF  =  500  ms,  some  effect  of  the  stimulus 
maybe  seen  as  early  as  -250  ms  (gray  bar  reflects 
true  pre-stimulus  period). 
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Figure  19  illustrates  that  PLI  in  the  two  lowest  frequency  bands  were  affected  by  motion  in  the 
ambulatory  condition  and  minimally  impacted  in  the  vehicle  motion  conditions  (see  mean  plus 
standard  deviation  above  zero).  PLI  in  the  higher  four  frequency  bands  appeared  to  be 
minimally,  if  at  all,  impacted  by  any  of  the  motion  conditions.  Importantly,  it  should  be  noted 
here  that  while  stride  frequency  and  the  movement  itself  in  these  conditions  are  generally 
associated  with  low  frequencies  (<delta  activity),  muscle  activity  associated  with  physical 
motion,  generally,  is  associated  with  higher  frequency  activity  (>  alpha  activity). 

The  previous  data  suggest  that  on  a  global  level,  across  the  entire  set  of  conditions  examined 
here,  PLI  appears  only  slightly  sensitive  to  lower  frequencies  and  appears  insensitive  to  the 
higher  frequencies.  The  question  thus  remains,  however,  is  PLI  sensitive  to  anything  in  the  tasks 
themselves  (previous  literature  has  suggested  that  PLI  is  sensitive  to  particular  disease 
states — Stam  et  al.  2009;  Stam  et  al  2007)?  To  address  this  question,  a  novel  event-related 
analysis  of  PLI  is  illustrated  in  figure  20.  In  this  figure,  time-evolving  PLI  is  depicting  starting 
approximately  2  s  prior  to  target  stimuli  onset.  Mean  values  at  each  time  step  are  depicted  with 
error  bars  reflecting  the  between  participants  standard  deviation.  A  pre-stimulus  baseline  is 
represented  from  -2.0  to  -0.25  s.  In  the  lowest  four  frequencies,  it  can  be  observed  that  generally 
there  is  more  variance  in  the  waveforms  after  the  onset  of  the  stimulus.  Further,  in  several  cases, 
the  mean  plus  standard  deviation  does  not  cross  the  baseline  boundary.  These  results  suggest  that 
PLI,  even  at  this  gross  level,  is  capable  of  reflecting  sensitivity  to  stimulus  processing. 

3.5  Conclusions 

The  primary  goal  of  this  project  is  to  enable  approaches  to  advance  our  current  understanding  of 
functional  brain  dynamics  in  operationally  relevant  settings  by  investigating  a  metric  that  (1)  can 
underlie  a  large-scale,  integrative  systems  approach,  (2)  has  sensitvity  to  neurocognitive 
processing,  and  (3)  has  low  sensitivity  to  signal  artifacts  generated  from  biological  and 
environmental  sources  while  recording  EEG  data  in  real-world  environments.  PLI  is  a  nonlinear 
measure  of  functional  connectivity  that  has  been  demonstrated  to  provide  the  basis  for  such  a 
large-scale,  integrative  approach  (Stam  et  al.,  2009).  Futher,  as  PLI  examines  only  non-zero 
phase  relationships,  it  is  purportedly  not  sensitive  to  volume  conduction  (Stam  et  al.,  2007).  The 
data  presented  here  are  generally  supportive  of  this  claim  and  provide  preliminary  evidence  that 
PLI  is  also  sensitive  to  basic  neurocognitive  processing. 

We  illustrate  that  on  a  global  level,  low-frequency  PLI  has  slight  sensitivity  to  self-generated 
motion  (i.e.,  walking  and  jogging);  however,  mid-  and  high-frequency  PLI  do  not  show  such 
sensitivity  to  any  motion  condition.  The  slight  sensitivity  of  PLI  in  the  lower  frequencies  could 
reflect  either  the  neurocognitive  processing  associated  with  self-generated  motion  or  could 
reflect  artifacts  from  these  conditions.  Importantly,  as  the  higher  frequency  PLIs  do  not  reflect 
such  sensitivity,  it  seems  unlikely  that  the  lower  frequency  PLIs  are  directly  reflecting  muscle 
activity.  Overall,  these  results  suggest  that  PLI  can  be  used  to  reliably  examine  brain  electrical 
data  in  certain  operational  environments  such  as  driving.  At  this  time,  careful  interpretation  is 
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necessary  in  ambulatory  vehicle  motion  environments.  A  more  specific,  network-based  analyses 
of  the  data  presented  here  may  provide  insights  into  the  source  of  the  apparent  low  frequency 
sensitivity  and  potentially  enable  the  broader  application  of  the  PLI  metric. 

We  further  illustrate  that  aspects  of  PLI  (i.e.,  either  the  PLI  value  or  its  variability)  appear 
sensitive  to  short-duration  neurocognitive  processing.  Considering  (1)  the  variability  in 
estimating  phase  in  the  500-ms  RF  here  and  (2)  the  very  general  nature  of  this  exploratory 
analysis,  this  result  is  very  promising  and  supports  the  investigation  of  PLI  in  a  large-scale, 
integrative  systems  approach. 

The  positive  results  presented  here  support  a  novel  approach  to  overcoming  artifact  issues  and 
present  alternative  perspectives  on  understanding  brain  behavior.  Such  results  here  give  hope 
that  PLI  can  be  applied  within  military-relevant  environments  and  augment  Soldier  state  and 
neurocognition  monitoring  technologies.  The  use  of  a  nonlinear  metric  of  functional  connectivity 
also  holds  the  promise  of  novel  real-time  insights  that  can  be  used  to  drive  and  improve  current 
conceptions  of  brain-machine  adaptive  training  and  operational  systems  and  in-field 
neurocognitive  assessment. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


2-D 

two-dimensional 

3-D 

three-dimensional 

6-DOF 

6  degree-of-freedom 

AFE 

analog  front  end 

ARO 

Army  Research  Office 

BOLD 

blood-oxygen-level  dependence 

CMOS 

complementary  metal-oxide  semiconductor 

CMRR 

common-mode  rejection  ratio 

CMS 

common  mode  sense 

DAQ 

data  acquisition 

DDA 

differential  difference  amplifiers 

DISALT 

Dismounted  Infantryman  Survivability  and  Lethality  Testbed 

ECG 

electrocardiogram 

EEG 

electroencephalogram 

EMG 

electromyogram 

EO 

eyes  open 

ERPs 

event-related  potentials 

I'MRI 

functional  magnetic  resonance  imaging 

fNIRS 

functional  near  infrared  spectroscopic 

EO 

input/output 

ICA 

Independent  Components  Analysis 

ICs 

integrated  circuits 

LED 

light-emitting  diode 

M 

Math 
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MEMS 

Micro-electromechanical  system 

MR 

magnetic  resonance 

MUX 

multiplexer 

NIR 

near  infrared 

OTA 

operational  transconductance  amplifier 

PCB 

printed  circuit  board 

PDs 

photodiodes 

PLI 

phase  lag  index 

PLS 

partial  least  squares 

PMOS 

positive  metal-oxide  semiconductor 

PPG 

photoplethysmography 

PSD 

power  spectral  density 

PSRR 

power  supply  rejection  ratio 

RF 

reference 

RMS 

ride  motion  simulator 

S 

shooting 

S&M 

shooting  and  math 

SVM 

Support  Vector  Machines 

UAV 

unmanned  aerial  vehicle 

VLSI 

very  large-scale  integration 
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